System and method for providing prediction models for predicting changes to placeholder values

ABSTRACT

The present disclosure relates to a method and non-transitory machine-readable storage medium encoded with instructions for using prediction models for predicting values, the medium comprising instructions for receiving a plurality of entry identifiers, instructions for receiving a value for each of the plurality of entry identifiers, instructions for determining whether the value for each of the plurality of entry identifiers has changed and a magnitude of the change, instructions for building a model for predicting a time-to-value change;, instructions for building a model for predicting a future magnitude of change, instructions for performing a simulation using the model for predicting the time-to-value change and the model for predicting the future magnitude of change and instructions for outputting a confidence interval.

CROSS-REFERENCE TO PRIOR APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/507957, filed on 18 May 2017. This application is hereby incorporatedby reference herein.

TECHNICAL FIELD

The present disclosure pertains to a system and method for providingprediction models for predicting changes to placeholder values.

BACKGROUND

Data analysis systems may facilitate presentation of real-time data andanalysis of data. Although computer-assisted real-time data analysissystems exist, such systems may deliberately avoid consideration ofcertain data, such as data for a current period and/or a latest periodthat is still subject to change. Thus, such systems may be prone tosignificant errors caused by a lag period between first entry and finalsettlement of key performance indicators in real-time. These and otherdrawbacks exist.

SUMMARY

Various embodiments described herein relate to a system configured toprovide stable predictions by effectuating prediction models forpredicting placeholder values. The system comprises one or moreprocessors configured by machine readable instructions and/or othercomponents. The system is configured to obtain historical informationrelated to one or more metrics. The historical information includes (i)placeholder values that are known to unlikely represent actual valuesfor which the placeholder values serve as placeholders, (ii) for each ofthe placeholder values, one or more updated placeholder values that arerevisions of the placeholder value, and (iii) timing information relatedto when the placeholder values are provided and when the updatedplaceholder values are respectively provided as revisions of theplaceholder values. The system is configured to generate predictionmodels based on the placeholder values, the updated placeholder values,and the timing information, such that, with respect to a latest updatedplaceholder value for each of the placeholder values, at least one ofthe prediction models is configured to generate a prediction related toa potential further revision to the latest updated placeholder valuewithin a given time window. The system is configured to generate one ormore predictions based on the prediction models, the predictions beingrelated to potential further revisions respective to the latest updatedplaceholder values. The system is configured to effectuate, via a userinterface, presentation of the one or more predictions.

Various embodiments described herein relate to a method for providingstable predictions by effectuating prediction models for predictingplaceholder values with a system. The system comprises one or moreprocessors configured by machine readable instructions and/or othercomponents. The method comprises obtaining, with the one or moreprocessors, historical information related to one or more metrics. Thehistorical information includes (i) placeholder values that are known tounlikely represent actual values for which the placeholder values serveas placeholders, (ii) for each of the placeholder values, one or moreupdated placeholder values that are revisions of the placeholder value,and (iii) timing information related to when the placeholder values areprovided and when the updated placeholder values are respectivelyprovided as revisions of the placeholder values. The method comprisesgenerating, with the one or more processors, prediction models based onthe placeholder values, the updated placeholder values, and the timinginformation, such that, with respect to a latest updated placeholdervalue for each of the placeholder values, at least one of the predictionmodels is configured to generate a prediction related to a potentialfurther revision to the latest updated placeholder value within a giventime window. The method comprises generating, with the one or moreprocessors, one or more predictions based on the prediction models, thepredictions being related to potential further revisions respective tothe latest updated placeholder values. The method compriseseffectuating, via a user interface, presentation of the one or morepredictions.

Various embodiments described herein relate to a system for providingstable predictions by effectuating prediction models for predictingplaceholder values. The system comprises means for obtaining historicalinformation related to one or more metrics The historical informationincludes (i) placeholder values that are known to unlikely representactual values for which the placeholder values serve as placeholders,(ii) for each of the placeholder values, one or more updated placeholdervalues that are revisions of the placeholder value, and (iii) timinginformation related to when the placeholder values are provided and whenthe updated placeholder values are respectively provided as revisions ofthe placeholder values. The system comprises means for generatingprediction models based on the placeholder values, the updatedplaceholder values, and the timing information, such that, with respectto a latest updated placeholder value for each of the placeholdervalues, at least one of the prediction models is configured to generatea prediction related to a potential further revision to the latestupdated placeholder value within a given time window. The systemcomprises means for generating one or more predictions based on theprediction models, the predictions being related to potential furtherrevisions respective to the latest updated placeholder values. Thesystem comprises means for effectuating presentation of the one or morepredictions.

Various embodiments described herein relate to a non-transitorymachine-readable storage medium for using prediction models forpredicting values, the medium including instructions for receiving aplurality of entry identifiers, instructions for receiving a value foreach of the plurality of entry identifiers, instructions for determiningwhether the value for each of the plurality of entry identifiers haschanged and a magnitude of the change, instructions for building a modelfor predicting a time-to-value change, instructions for building a modelfor predicting a future magnitude of change, instructions for performinga simulation using the model for predicting the time-to-value change andthe model for predicting the future magnitude of change and instructionsfor outputting a confidence interval.

In an embodiment of the present disclosure, the non-transitorymachine-readable storage medium for using prediction models forpredicting values, the medium comprising instructions for defining anentry time, the entry time being a first time when at least one of theplurality of entry identifiers was received.

In an embodiment of the present disclosure, the non-transitorymachine-readable storage medium for using prediction models forpredicting values, the medium comprising instructions for defining atime to change, the time to change being a lag time between when atleast one of the plurality of entry identifiers was received and when achange in the value for at least one of the plurality of entryidentifiers was received.

In an embodiment of the present disclosure, the time to change isdefined as

D = {d₁, … , d_(n_(t_(max)))}

where d_(m)=min(argmax_(i)c_(m)i, t_(max))−e_(m).

In an embodiment of the present disclosure, the change, C^(i) isdetermined by

C^(i) = {c_(p_(j)^(i))}

where

$c_{p_{j}}^{i} = \left\{ {\begin{matrix}1 & {{{if}\mspace{14mu} v_{p_{j}^{i}}} \neq v_{p_{j}^{i + 1}}} \\0 & {else}\end{matrix},} \right.$

where v is the value for each of the plurality of entry identifiers.

In an embodiment of the present disclosure, the model for predicting atime-to-change model is a Poisson process model.

In an embodiment of the present disclosure, the model for predicting atime-to-change model is a Cox regression model.

In an embodiment of the present disclosure, the model for predicting thetime-to-change outputs a probability that the value for each of theplurality of entry identifiers will change.

In an embodiment of the present disclosure, the model for predicting thefuture magnitude of change is a first order statistic model of themagnitude of change.

In an embodiment of the present disclosure, the magnitude of change, ΔVis defined as ΔV={v_(p)(t_(m)+d_(m))−v_(p)t_(m)} for each of theplurality of entry identifiers with c_(p)=1 in any C^(i), where v is thevalue for each of the plurality of entry identifiers, c is the changeand t is the time value.

In an embodiment of the present disclosure, the simulation using themodel for predicting the time-to-value change and the model forpredicting the future magnitude of change is a Monte Carlo simulation.

In an embodiment of the present disclosure, the simulation changes themagnitude of change using the probability that the value for each of theplurality of entry identifiers will change and the magnitude of change.

Various embodiments described herein relate to a method for providingpredictions using prediction models for predicting values, the methodincluding the steps of receiving a plurality of entry identifiers,receiving a value for each of the plurality of entry identifiers,determining whether the value for each of the plurality of entryidentifiers has changed and a magnitude of the change, building a modelfor predicting a time-to-value change, building a model for predicting afuture magnitude of change, performing a simulation using the model forpredicting the time-to-value change and the model for predicting thefuture magnitude of change and outputting a confidence interval.

In an embodiment of the present disclosure, the method for providingpredictions using prediction models for predicting values, the methodfurther including the step of defining an entry time, the entry timebeing a first time when at least one of the plurality of entryidentifiers was received.

In an embodiment of the present disclosure, the method for providingpredictions using prediction models for predicting values, the methodfurther including the step of defining a time to change, the time tochange being a lag time between when at least one of the plurality ofentry identifiers was received and when a change in the value for atleast one of the plurality of entry identifiers was received.

In an embodiment of the present disclosure, the time to change isdefined as

D = {d₁, …  , d_(n_(t_(max)))}

where d_(m)=min(argmax_(i)c_(m)i, t_(max))−e_(m).

In an embodiment of the present disclosure, the change, C^(i) isdetermined by

C^(i) = {c_(p_(j)^(i))}

where

$c_{p_{j}^{i}} = \left\{ {\begin{matrix}1 & {{{if}\mspace{14mu} v_{p_{j}^{i}}} \neq v_{p_{j}^{i + 1}}} \\0 & {else}\end{matrix},} \right.$

where v is the value for each of the plurality of entry identifiers.

In an embodiment of the present disclosure, the model for predicting atime-to-change model is a Poisson process model.

In an embodiment of the present disclosure, the model for predicting atime-to-change model is a Cox regression model.

In an embodiment of the present disclosure, the model for predicting thetime-to-change outputs a probability that the value for each of theplurality of entry identifiers will change.

In an embodiment of the present disclosure, the model for predicting thefuture magnitude of change is a first order statistic model of themagnitude of change.

In an embodiment of the present disclosure, the future magnitude ofchange, ΔV is defined as ΔV={v_(p)(t_(m)+d_(m))−v_(p)t_(m)} for each ofthe plurality of entry identifiers with c_(p)=1 in any C^(i), where v isthe value for each of the plurality of entry identifiers, c is thechange and t is the time value.

In an embodiment of the present disclosure, the simulation using themodel for predicting the time-to-value change and the model forpredicting the future magnitude of change is a Monte Carlo simulation.

In an embodiment of the present disclosure, the simulation changes themagnitude of change using the a probability that the value for each ofthe plurality of entry identifiers will change and the magnitude ofchange.

These and other objects, features, and characteristics of the presentdisclosure, as well as the methods of operation and functions of therelated elements of structure and the combination of parts and economiesof manufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system configured to provide prediction models forpredicting changes to placeholder values, in accordance with one or moreembodiments.

FIG. 2 illustrates data extraction at subsequent moments in time, inaccordance with one or more embodiments.

FIG. 3 illustrates a method for providing prediction models forpredicting changes to placeholder values, in accordance with one or moreembodiments.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

As used herein, the singular form of “a”, “an”, and “the” include pluralreferences unless the context clearly dictates otherwise. As usedherein, the term “or” means “and/or” unless the context clearly dictatesotherwise. As used herein, the statement that two or more parts orcomponents are “coupled” shall mean that the parts are joined or operatetogether either directly or indirectly, i.e., through one or moreintermediate parts or components, so long as a link occurs. As usedherein, “directly coupled” means that two elements are directly incontact with each other. As used herein, “fixedly coupled” or “fixed”means that two components are coupled so as to move as one whilemaintaining a constant orientation relative to each other.

As used herein, the word “unitary” means a component is created as asingle piece or unit. That is, a component that includes pieces that arecreated separately and then coupled together as a unit is not a“unitary” component or body. As employed herein, the statement that twoor more parts or components “engage” one another shall mean that theparts exert a force against one another either directly or through oneor more intermediate parts or components. As employed herein, the term“number” shall mean one or an integer greater than one (i.e., aplurality).

Directional phrases used herein, such as, for example and withoutlimitation, top, bottom, left, right, upper, lower, front, back, andderivatives thereof, relate to the orientation of the elements shown inthe drawings and are not limiting upon the claims unless expresslyrecited therein.

Data quality may include several elements, including availability ofdata (e.g., whether data is missing), measurement quality (e.g., marginof measurement error), whether the data is final or will be subject tochange in (near) future, and/or other elements. As an example,healthcare claims data (e.g., costs, amounts reimbursed, and/or otherdata) may be subject to change after first entry of the data and beforereaching final settlement. As such, the settlement period may persistfor one or more months, one or more years, and/or other periods.

In one scenario, for instance, medical insurance claims may undergo oneor more stages of processing during which payment values are subject tochange. In this example, medical insurance claims may be:

-   -   incurred but not reported (IBNR) yet by the hospital (or        beneficiary) in which the insurer does not yet know about the        existence of the claim and the claimed amount but the hospital        (or beneficiary) has already incurred the costs. As such, there        is a lag time between incurred costs and reported claims in        which a hospital expects a reimbursement for its cost made and        an insurer needs to make an estimate on the not-yet-reported        claims to reserve a budget for them;    -   incurred, reported but not settled (RBNS) yet in which a first        payment from the insurer to the hospital is made but not settled        yet to a final amount. As such, there is a lag time between a        first initial payment of a reported claims and its settlement of        a final payment, which may result in various changing payments        of the reported claim until settlement over time;    -   incurred, reported, but not paid (RBNP) in which the hospital        has reported the claim to the insurer, but the insurer has not        decided yet on payment. As such, there is a lag time between a        reported claim and its payment; or    -   paid in which the claim has received a final payment from the        insurer to the hospital; this payment may or may not differ from        the claimed or incurred amount.

Systems currently available may omit data obtained during a currentperiod and/or a latest period (e.g., last six months) in order to avoiddata that is still subject to change. Other systems may assume that thedata indicated as being subject to change includes finalized and correctdata. However, costs and reimbursement amounts associated with the samepatient and/or treatment may change dramatically from month to month.Such a change may be caused by incorrect data input, insurernegotiations, court-cases, claim settlements and various types ofcorrections. Changes to costs and reimbursement amounts maysignificantly influence data summaries such as mean, median, standarddeviation, inter-quartile-range, and/or other statistics.

FIG. 1 is a schematic illustration of a system 10 configured to provideprediction models for predicting changes to placeholder values, inaccordance with one or more embodiments. In some embodiments, system 10is configured to obtain historical information corresponding to (i)placeholder values that are known to unlikely represent actual values(for which the placeholder values serve as placeholders), (ii) updatedplaceholder values that are revisions of the placeholder value, and(iii) timing information related to the placeholder values and therevisions. As an example, a placeholder value may include an estimatedor other placeholder value that is subject to change (e.g., given thatit is known that the placeholder value is unlikely an accuraterepresentation of the final settlement value). In some embodiments,system 10 is configured to generate a prediction related to a potentialrevision to a placeholder value within a given time window. In someembodiments, system 10 is configured to effectuate presentation of adescriptive statistic (e.g., mean, median, mode, standard deviation,kurtosis, skewness, and/or other statistics) and a confidence interval(e.g., a range of values) corresponding to the prediction. For example,based on the stability of placeholder values in the historicalinformation, we are 95% confident that the true descriptive statistic iswithin a determined range of values.

For example, system 10 may facilitate information retrieval overmultiple extractions (e.g., batches or updates) of the same data and/ordata from the same dynamic cohort (e.g., as patients may enter or leavethe cohort because of inclusion or discontinuation of the program) takenat different points in time. As such, system 10 facilitates determiningone or more of a probability of one or more changes in existing datapoints, a magnitude of one or more changes in existing data points, aprediction related to potential changes to existing data points, orother information.

In some embodiments, system 10 facilitates predicting a final value ofan insurance claim settlement based on historical information. Forexample, in connection with the previously described medical insuranceclaims status, an insurer may, based on historical claim settlementinformation, (i) determine a final amount for an insurance claim and(ii) provide final payment to a healthcare facility and/or care providerand forgo a settlement period in which various changing payments of thereported claim occur over time until the claim has been settled. Assuch, system 10 may facilitate providing updated informationcorresponding to one or more metrics with more accuracy (e.g., byincorporating historical information inclusive of a current period oftime) and less time lag.

In some embodiments, system 10 comprises one or more processors 12,electronic storage 14, external resources 16, computing device 18, orother components.

Electronic storage 14 comprises electronic storage media thatelectronically stores information (e.g., criteria, mathematicalequations, predictions, etc.). The electronic storage media ofelectronic storage 14 may comprise one or both of system storage that isprovided integrally (i.e., substantially non-removable) with system 10and/or removable storage that is removably connectable to system 10 via,for example, a port (e.g., a USB port, a firewire port, etc.) or a drive(e.g., a disk drive, etc.). Electronic storage 14 may be (in whole or inpart) a separate component within system 10, or electronic storage 14may be provided (in whole or in part) integrally with one or more othercomponents of system 10 (e.g., computing device 18, processor 12, etc.).In some embodiments, electronic storage 14 may be located in a servertogether with processor 12, in a server that is part of externalresources 16, in a computing device 18, and/or in other locations.Electronic storage 14 may comprise one or more of optically readablestorage media (e.g., optical disks, etc.), magnetically readable storagemedia (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.),electrical charge-based storage media (e.g., EPROM, RAM, etc.),solid-state storage media (e.g., flash drive, etc.), and/or otherelectronically readable storage media. Electronic storage 14 may storesoftware algorithms, information determined by processor 12, informationreceived via computing devices 18 and/or graphical user interface 20and/or other external computing systems, information received fromexternal resources 16, and/or other information that enables system 10to function as described herein.

External resources 16 include sources of information and/or otherresources. For example, external resources 16 may include medicalinsurance claim information and/or other information (e.g., medicalinsurance claims archive of an insurer). In some embodiments, externalresources 16 include health information related to a patient. In someembodiments, the health information comprises demographic information,vital signs information, medical condition information indicatingmedical conditions experienced by the patient, treatment informationindicating treatments received by the patient, outcome informationindicating health outcomes for the patient, and/or other healthinformation. In some embodiments, external resources 16 include sourcesof information such as databases, websites, etc., external entitiesparticipating with system 10 (e.g., a medical records system of a healthcare provider that stores medical history information for populations ofpatients), one or more servers outside of system 10, and/or othersources of information. In some embodiments, external resources 16include components that facilitate communication of information such asa network (e.g., the internet), electronic storage, equipment related toWi-Fi technology, equipment related to Bluetooth® technology, data entrydevices, sensors, scanners, and/or other resources. External resources16 may be configured to communicate with processor 12, computing device18, electronic storage 14, and/or other components of system 10 viawired and/or wireless connections, via a network (e.g., a local areanetwork and/or the internet), via cellular technology, via Wi-Fitechnology, and/or via other resources. In some embodiments, some or allof the functionality attributed herein to external resources 16 may beprovided by resources included in system 10.

Computing devices 18 are configured to provide an interface between user34 (e.g., insurance claim processors, hospital billing staff, doctors,nurses, administrators, staff members, technicians, etc.), and/or otherusers, and system 10. In some embodiments, individual computing devices18 are and/or are included in desktop computers, laptop computers,tablet computers, smartphones, and/or other computing devices associatedwith individual caregivers 14, individual patients 12, and/or otherusers. In some embodiments, individual computing devices 18 are, and/orare included in equipment used in insurer's offices, hospitals, doctor'soffices, and/or other facilities. Computing devices 18 are configured toprovide information to and/or receive information from user 34, and/orother users. For example, computing devices 18 are configured to presenta graphical user interface 20 to user 34 to facilitate entry and/orselection of a descriptive statistic and a margin of error (e.g., asdescribed below). In some embodiments, graphical user interface 20includes a plurality of separate interfaces associated with computingdevices 18, processor 12, and/or other components of system 10; multipleviews and/or fields configured to convey information to and/or receiveinformation from user 34, and/or other users; and/or other interfaces.

In some embodiments, computing devices 18 are configured to provide userinterface 20, processing capabilities, databases, or electronic storageto system 10. As such, computing devices 18 may include processor 12,electronic storage 14, external resources 16, or other components ofsystem 10. In some embodiments, computing devices 18 are connected to anetwork (e.g., the internet). In some embodiments, computing devices 18do not include processor 12, electronic storage 14, external resources16, or other components of system 10, but instead communicate with thesecomponents via the network. The connection to the network may bewireless or wired. For example, processor 12 may be located in a remoteserver and may wirelessly cause presentation of the one or morepredictions via the user interface to a care provider on computingdevices 18 associated with that caregiver (e.g., a doctor, a nurse, acentral caregiver coordinator, etc.). In some embodiments, computingdevices 18 are laptops, desktop computers, smartphones, tabletcomputers, or other computing devices.

Examples of interface devices suitable for inclusion in user interface20 include a camera, a touch screen, a keypad, touch sensitive orphysical buttons, switches, a keyboard, knobs, levers, a display,speakers, a microphone, an indicator light, an audible alarm, a printer,tactile haptic feedback device, or other interface devices. The presentdisclosure also contemplates that computing devices 18 includes aremovable storage interface. In this example, information may be loadedinto computing devices 18 from removable storage (e.g., a smart card, aflash drive, a removable disk, etc.) that enables caregivers or otherusers to customize the implementation of computing device 18. Otherexemplary input devices and techniques adapted for use with Computingdevices 18 or the user interface include an RS-232 port, RF link, an IRlink, a modem (telephone, cable, etc.), or other devices or techniques.

Processor 12 is configured to provide information processingcapabilities in system 10. As such, processor 12 may comprise one ormore of a digital processor, an analog processor, a digital circuitdesigned to process information, an analog circuit designed to processinformation, a state machine, or other mechanisms for electronicallyprocessing information. Although processor 12 is shown in FIG. 1 as asingle entity, this is for illustrative purposes only. In someembodiments, processor 12 may comprise a plurality of processing units.These processing units may be physically located within the same device(e.g., a server), or processor 12 may represent processing functionalityof a plurality of devices operating in coordination (e.g., one or moreservers, computing device 18, devices that are part of externalresources 16, electronic storage 14, or other devices.)

In some embodiments, processor 12, external resources 16, computingdevices 18, electronic storage 14, and/or other components may beoperatively linked via one or more electronic communication links. Forexample, such electronic communication links may be established, atleast in part, via a network such as the Internet, and/or othernetworks. It will be appreciated that this is not intended to belimiting, and that the scope of this disclosure includes embodiments inwhich these components may be operatively linked via some othercommunication media. In some embodiments, processor 12 is configured tocommunicate with external resources 16, computing devices 18, electronicstorage 14, and/or other components according to a client/serverarchitecture, a peer-to-peer architecture, and/or other architectures.

As shown in FIG. 1, processor 12 is configured via machine-readableinstructions 24 to execute one or more computer program components. Thecomputer program components may comprise one or more of a communicationscomponent 26, a model generation component 28, a prediction component30, a presentation component 32, or other components. Processor 12 maybe configured to execute components 26, 28, 30, or 32 by software;hardware; firmware; some combination of software, hardware, or firmware;or other mechanisms for configuring processing capabilities on processor12.

It should be appreciated that although components 26, 28, 30, and 32 areillustrated in FIG. 1 as being co-located within a single processingunit, in embodiments in which processor 12 comprises multiple processingunits, one or more of components 26, 28, 30, or 32 may be locatedremotely from the other components. The description of the functionalityprovided by the different components 26, 28, 30, or 32 described belowis for illustrative purposes, and is not intended to be limiting, as anyof components 26, 28, 30, or 32 may provide more or less functionalitythan is described. For example, one or more of components 26, 28, 30, or32 may be eliminated, and some or all of its functionality may beprovided by other components 26, 28, 30, or 32. As another example,processor 12 may be configured to execute one or more additionalcomponents that may perform some or all of the functionality attributedbelow to one of components 26, 28, 30, or 32.

Communications component 26 is configured to obtain historicalinformation related to one or more metrics. In some embodiments, the oneor more metrics include one or more of medical insurance claims, (re)admissions to a healthcare facility, investments, sales metrics,marketing metrics, supply chain metrics, retail metrics, social mediametrics, and/or other key performance indicators (KPIs). In someembodiments, the historical information includes (i) placeholder valuesthat are known to unlikely represent actual values for which theplaceholder values serve as placeholders, (ii) for each of theplaceholder values, one or more updated placeholder values that arerevisions of the placeholder value, and (iii) timing information relatedto when the placeholder values are provided and when the updatedplaceholder values are respectively provided as revisions of theplaceholder values. In some embodiments, placeholder values include oneor more values likely to be subject to change. The one or more valueslikely to be subject to change may not be known in advance. By way of anon-limiting example, the historical information obtained at time 0 mayinclude values corresponding to the one or more metrics. In thisexample, one or more values may be subject to change; however, adetermination as to which values are subject to change, when the valueschange, and/or how much (e.g., magnitude) the values change may not beascertained until one or more subsequent data sets have been obtained.In some embodiments, communications component 26 is configured tocontinuously obtain the historical information (e.g., on a periodicbasis, in accordance with a schedule, or based on other automatedtriggers). For example, communications component 26 is configured toobtain patient health records every month. In some embodiments,communications component is configured to continuously obtain furtherupdated placeholder values (e.g., on a periodic basis, in accordancewith a schedule, or based on other automated triggers).

By way of a non-limiting example, FIG. 2 illustrates historicalinformation corresponding to one or more patients obtained from ahospital electronic health record (EHR), in accordance with one or moreembodiments. FIG. 2 provides an illustration of data extraction atsubsequent moments in time. In FIG. 2, cells having diagonal-patternshading indicate (same) data presence and cells having solid-fillshading indicate changed data. As shown in FIG. 2, costs ‘a’ and ‘b’associated with patient 1 have been entered at a first time (e.g., time0). Furthermore, as evident in the data corresponding to the monthssubsequent to time 0, cost ‘b’ associated with patient 1 has beensettled upon at time 0; however, as indicated in time 2, a value of cost‘a’ associated with patient 1 undergoes a change.

In some embodiments, individual ones of the one or more metrics areassociated with one or more attributes. In some embodiments, a firstattribute and a second attribute are mutually associated with aplaceholder value and an updated placeholder value (e.g., that is arevised version of the placeholder value). For example, the firstattribute may include patient identification information, and the secondattribute may include cost identification information. In someembodiments, communications component 26 is configured to obtain aco-variate associated with the second attribute. In some embodiments,the co-variate includes one or more of a cost item type, principaldiagnosis of a user, a specific treatment group of the user, asocioeconomic status of the user, disease information associated withthe user, and/or other information. For example, the type of cost item(or reported claim) may be used as a covariate that is indicative ofwhether the cost has been incurred from using an inpatient, outpatientor pharmacy service. As another example, the principal diagnosis (Dx) orspecific treatment group (Rx) reported in the claim may be used as acovariate (e.g., in a Cox regression, the hazard rate ratio).

Returning to FIG. 1, model generation component 28 is configured togenerate prediction models based on the placeholder values, the updatedplaceholder values, the timing information, or other information. Insome embodiments, model generation component 28 is configured todetermine a first entry time for an individual one of the one or moremetrics in the historical information. In some embodiments, the firstentry time is indicative of a first occurrence of a combination of thefirst attribute and the second attribute. For example, let P^(i)={p^(i)1, . . . , p^(i)ni} be the set of n_(i) unique entry ids extracted attime i ϵ {0, . . . , t_(max)}. In connection with the exampleillustrated in FIG. 2, p may be indexed by the combination of patient-idand cost-id. Additional variables, other than cost-id and value, includeage of patient, disease state, institute visited, etc. In this example,the entry times E may be defined as, for each first occurrence of p_(j)^(i) in P^(i), E={e₁, . . . , e_(n) _(tmax) }, where t_(m)=argmin_(i)p^(i) _(m) □P^(i). As such, an entry time may indicate the first momentin time in which a combination of a particular patient with a particularcost item was present in a data extraction.

In some embodiments, model generation component 28 is configured todetermine a time to change for an individual one of the one or moremetrics in the historical information. In some embodiments, the time tochange is indicative of a time lag between the first occurrence of thecombination of the first attribute and the second attribute and a secondoccurrence of the combination of the first attribute and the secondattribute. In some embodiments, the first occurrence is associated witha placeholder value. In some embodiments, the second occurrence isassociated with an updated placeholder value. In some embodiments, theupdated placeholder value is different from the placeholder value.

For example, let

D = {d₁, …  , d_(n_(t_(max)))}

be the set of times to change for one or more entries, whereind_(m)=min(argmax_(i) c_(m)i, t_(max))−e_(m).

In some embodiments, model generation component 28 is configured todetermine one or more change indicators based on a comparison of asuccessive pair of placeholder values mutually associated with the firstattribute and the second attribute. In some embodiments, the changeindicators include binary (e.g. 1 if there is a change and 0 if there isno change) change indicators. In some embodiments, the successive pairof placeholder values includes a placeholder value and an updatedplaceholder value, an updated placeholder value and a further updatedplaceholder value, and/or other successive values. For example, letV^(i)={v_(p)i} be the values assigned to data entry p^(i). In thisexample, a set of change-indicators C^(i) may be defined as

C^(i) = {c_(p_(j)^(i))}

where

$c_{p_{j}^{i}} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} v_{p_{j}^{i}}} \neq v_{p_{j}^{i + 1}}} \\0 & {else}\end{matrix} \right.$

In some embodiments, model generation component 28 is configured suchthat, with respect to a latest updated placeholder value for each of theplaceholder values, at least one of the prediction models is configuredto generate a prediction related to a potential further revision to thelatest updated placeholder value within a given time window. In someembodiments, model generation component 28 is configured such that theprediction models include, with respect to a latest updated placeholdervalue for at least one of the placeholder values, a first predictionmodel configured to generate a prediction related to a probability of apotential further revision to the latest updated placeholder valuewithin a given time window. For example, model generation component 28is configured to use the first entry time and the time to change in asurvival (time-to-event) analysis. In some embodiments, the predictionmodel includes one or more of a Poisson process model, a Cox regressionmodel, and/or other models. In these models, t_(max) may be used as(right-) censoring time. In some embodiments, an output of the firstprediction model may be indicative of a probability p_(change) of achange in the value (e.g., the placeholder value, the updatedplaceholder value, and/or other values) within a particular time window.In some embodiments, model generation component 28 is configured suchthat the predictive model takes into account characteristics of the oneor more metrics to create a likelihood (e.g., a conditional probabilitybased on the characteristics). In some embodiments, the characteristicsinclude the co-variate obtained via communications component 26 (e.g.,as described above). In some embodiments, machine learning may be usedto generate probabilities using neural networks, support vectorregression or other machine learning techniques.

In some embodiments, model generation component 28 is configured suchthat the prediction models include, with respect to a latest updatedplaceholder value for at least one of the placeholder values, a secondprediction model configured to generate a prediction related to amagnitude of a potential further revision to the latest updatedplaceholder value within a given time window. For example, letΔV={v_(p)(t_(m)+d_(m))−v_(p)t_(m)} be a set of magnitudes of the changein value in case a change happens. In this example ΔV may be defined forall p with c_(p)=1 in any C^(i). In some embodiments, the predictionmodel includes one or more of a first order statistic (e.g., a mean or amedian) of ΔV, a linear regression model that predicts ΔV based uponcharacteristics assigned (including but not limited to v) to the dataentries represented in ΔV, and/or other models.

In some embodiments, model generation component 28 is configured to,based on a third prediction model, determine the probability of a changefor the next data extraction. In some embodiments, the third predictionmodel includes a logistic regression model and/or other models. Forexample, model generation component 28 may facilitate determining avalue corresponding to an updated placeholder, a further updatedplaceholder, and/or other values given a fixed time horizon. In thisexample, the fixed time horizon may include a month, two months, oneyear, and/or other future times from a current time.

In some embodiments, model generation component 28 is configured togenerate a fourth prediction model based on recurrent changes if thedata are expected to change in value more than once. In someembodiments, the fourth prediction model includes the Anderson-Gillmethod and/or other models. For example, model generation component 28may facilitate determining a number of readmissions to a healthcarefacility during a predetermined period (e.g., 30 days) and costsassociated with the recurring readmissions for a patient.

In some embodiments, one or more of the first prediction model, thesecond prediction model, the third prediction model, or the fourthprediction model may be and/or include a support vector regression,decision trees/forests, a regression learning vector quantization, ak-nearest neighbor regression, and/or other models.

In some embodiments, one or more of the first prediction model, thesecond prediction model, the third prediction model, or the fourthprediction model may be and/or include a neutral network that is trainedand utilized for generating predictions (described below). As anexample, neural networks may be based on a large collection of neuralunits (or artificial neurons). Neural networks may loosely mimic themanner in which a biological brain works (e.g., via large clusters ofbiological neurons connected by axons). Each neural unit of a neuralnetwork may be connected with many other neural units of the neuralnetwork. Such connections can be enforcing or inhibitory in their effecton the activation state of connected neural units. In some embodiments,each individual neural unit may have a summation function which combinesthe values of all its inputs together. In some embodiments, eachconnection (or the neutral unit itself) may have a threshold functionsuch that the signal must surpass the threshold before it is allowed topropagate to other neural units. These neural network systems may beself-learning and trained, rather than explicitly programmed, and canperform significantly better in certain areas of problem solving, ascompared to traditional computer programs. In some embodiments, neuralnetworks may include multiple layers (e.g., where a signal pathtraverses from front layers to back layers). In some embodiments, backpropagation techniques may be utilized by the neural networks, whereforward stimulation is used to reset weights on the “front” neuralunits. In some embodiments, stimulation and inhibition for neuralnetworks may be more free-flowing, with connections interacting in amore chaotic and complex fashion.

In some embodiments, model generation component 28 is configured toupdate the prediction models based on (i) at least some further updatedplaceholder values that are revisions respectively to at least some ofthe updated placeholder values, (ii) further timing information relatedto when the one or more further updated placeholder values arerespectively provided as revisions of the at least some updatedplaceholder values, or (iii) other information (e.g., other historicalinformation described herein). In some embodiments, model generationcomponent 28 is configured to automatically update the prediction modelsresponsive to updated placeholder values, further updated placeholdervalues, and/or other information being obtained. For example, theprediction models may be updated such that values corresponding toupdated placeholders (e.g., values present in time 2) are replaced withvalues corresponding to further updated placeholder values (e.g., valuesto be received in time 3). As another example, the prediction models maybe used to generate predictions using placeholder values that are knownto unlikely represent actual values such that data for a current periodand/or a latest period is incorporated in determining the predictionmodels. In this example, responsive to obtaining updated placeholdervalues that are revisions of the placeholder value, model generationcomponent 28 is configured to update the prediction models with theupdated placeholder values to facilitate more accurate predictions. Assuch, model generation component 28 is configured to provide dynamicprediction models in which updated placeholder values (e.g., values thatare more likely to represent actual values) are used to update theprediction models in order to (i) automatically detect the level ofstability of a dataset, (ii) provide predictions for the uncertainty dueto instability of data, and (iii) provide predictions using more stabledata.

Prediction component 30 is configured to generate one or morepredictions based on the prediction models. In some embodiments, thepredictions are related to potential further revisions respective to thelatest updated placeholder values. In some embodiments, predictioncomponent 30 is configured to determine a confidence interval for adescriptive statistic (e.g., mean, median, mode, standard deviation,kurtosis, skewness, and/or other statistics) corresponding to anindividual one of the one or more metrics based on the first model andthe second model. In some embodiments, prediction component 30 isconfigured to determine the confidence interval based on a Monte Carlosimulation of the first model and the second model. For example, anaverage value of V^(i) in month i is calculated as

$\frac{1}{n}\Sigma \; {v_{i}.}$

In this example, a Monte Carlo simulation may be used to change thevalues in V^(i) with probability p_(change) and magnitude ΔV. In someembodiments, the determination of the average value V^(i) and the MonteCarlo simulation is repeated for a predetermined number of times suchthat a distribution of the statistic is obtained. In some embodiments,prediction component 30 is configured to determine a confidence intervaland/or other measures of margin-of-error for the statistic based on thedetermined distribution. In some embodiments, prediction component 30 isconfigured to automatically generate one or more predictions responsiveto a determination that a placeholder value is unlikely to representactual values for which the placeholder value serves as a placeholder(e.g., data corresponding to a current period, data undergoing recurrentchanges, and/or other information).

In order to provide predictions using prediction models for predictingvalues, the method identifies when a change occurs to the values andwith what magnitude. After the change is determined, a model is builtfor predicting time-to-value change. After building the model forpredicting the time-to-value change, a model is built for predicting amagnitude of change. After building the time-to-value change model andthe magnitude of change model, these two models are used in a MonteCarlo simulation to derive a margin-of-error for the statistic (i.e. aconfidence value).

Presentation component 32 is configured to effectuate, via userinterface 20, presentation of the one or more predictions. In someembodiments, presentation component 32 is configured to effectuate, viauser interface 20, presentation of the descriptive statistic and theconfidence interval. As such, presentation component 32 may facilitatedetermination of an influence of data changes on the descriptivestatistic. In some embodiments, presentation component 32 may effectuatepresentation of the one or more predictions, the descriptive statistic,the confidence interval, and/or other information via a webpage, aclinical dashboard, a spreadsheet (e.g., Excel), a statistical softwarepackage (e.g., SPSS, STATA) and/or other interfaces. For example,presentation component 32 may effectuate presentation of the one or morepredictions, the descriptive statistic, the confidence interval, and/orother information via a scatter plot, a chart, a histogram, a table,and/or other plots. In connection with the previously described medicalinsurance claims status, hospital administration may be presented with apredicted final payment value from an insurer for one or more medicalinsurance claims (e.g., medical insurance claims previously submitted bythe hospital) along with a measure of margin of error based onhistorical information corresponding to the hospital's current andprevious medical claim settlements. As another example, a patientseeking a particular treatment may be presented with a predictedreimbursement amount from an insurer based on previous treatmentsreceived by the patient (e.g., for similar treatments and/or differenttreatments) or other patients (e.g., other patients currently receivingsimilar treatments and/or patients who previously received similartreatments). Presentation component 32 may be configured to facilitateselection by user 34 and/or other users of the plots and/or theinformation presented via user interface 20. In some embodiments,presentation component 32 is configured to automatically effectuatepresentation of a predetermined descriptive statistic and apredetermined measure of margin of error.

FIG. 3 illustrates a method 300 for providing prediction models forpredicting changes to placeholder values with a system. The systemcomprises one or more processors and/or other components. The one ormore processors are configured by machine readable instructions toexecute computer program components. The computer program componentsinclude a communications component, a model generation component, aprediction component, a presentation, and/or other components. Theoperations of method 300 presented below are intended to beillustrative. In some embodiments, method 300 may be accomplished withone or more additional operations not described, and/or without one ormore of the operations discussed. Additionally, the order in which theoperations of method 300 are illustrated in FIG. 3 and described belowis not intended to be limiting.

In some embodiments, method 300 may be implemented in one or moreprocessing devices (e.g., a digital processor, an analog processor, adigital circuit designed to process information, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information). The one or moreprocessing devices may include one or more devices executing some or allof the operations of method 300 in response to instructions storedelectronically on an electronic storage medium. The one or moreprocessing devices may include one or more devices configured throughhardware, firmware, and/or software to be specifically designed forexecution of one or more of the operations of method 300.

At an operation 302, historical information related to one or moremetrics is obtained. In some embodiments, the historical informationincludes (i) placeholder values that are known to unlikely representactual values for which the placeholder values serve as placeholders,(ii) for each of the placeholder values, one or more updated placeholdervalues that are revisions of the placeholder value, and (iii) timinginformation related to when the placeholder values are provided and whenthe updated placeholder values are respectively provided as revisions ofthe placeholder values. In some embodiments, operation 302 is performedby a processor component the same as or similar to communicationscomponent 26 (shown in FIG. 1 and described herein).

At an operation 304, prediction models are generated based on theplaceholder values, the updated placeholder values, and the timinginformation, such that, with respect to a latest updated placeholdervalue for each of the placeholder values, at least one of the predictionmodels is configured to generate a prediction related to a potentialfurther revision to the latest updated placeholder value within a giventime window. In some embodiments, operation 304 is performed by aprocessor component the same as or similar to model generation component28 (shown in FIG. 1 and described herein).

At an operation 306, one or more predictions are generated based on theprediction models. In some embodiments, the predictions are related topotential further revisions respective to the latest updated placeholdervalues. In some embodiments, operation 306 is performed by a processorcomponent the same as or similar to prediction component 30 (shown inFIG. 1 and described herein).

At an operation 308, the one or more predictions are presented. In someembodiments, operation 308 is caused by a processor component the sameas or similar to presentation component 32 (shown in FIG. 1 anddescribed herein).

In the claims, any reference signs placed between parentheses shall notbe construed as limiting the claim. The word “comprising” or “including”does not exclude the presence of elements or steps other than thoselisted in a claim. In a device claim enumerating several means, severalof these means may be embodied by one and the same item of hardware. Theword “a” or “an” preceding an element does not exclude the presence of aplurality of such elements. In any device claim enumerating severalmeans, several of these means may be embodied by one and the same itemof hardware. The mere fact that certain elements are recited in mutuallydifferent dependent claims does not indicate that these elements cannotbe used in combination.

Although the description provided above provides detail for the purposeof illustration based on what is currently considered to be the mostpractical and preferred embodiments, it is to be understood that suchdetail is solely for that purpose and that the disclosure is not limitedto the expressly disclosed embodiments, but, on the contrary, isintended to cover modifications and equivalent arrangements that arewithin the spirit and scope of the appended claims. For example, it isto be understood that the present disclosure contemplates that, to theextent possible, one or more features of any embodiment can be combinedwith one or more features of any other embodiment.

What is claimed is:
 1. A non-transitory machine-readable storage mediumfor using prediction models for predicting values, the mediumcomprising: instructions for receiving a plurality of entry identifiers;instructions for receiving a value for each of the plurality of entryidentifiers; instructions for determining whether the value for each ofthe plurality of entry identifiers has changed and a magnitude of thechange; instructions for building a model for predicting a time-to-valuechange; instructions for building a model for predicting a futuremagnitude of change; instructions for performing a simulation using themodel for predicting the time-to-value change and the model forpredicting the future magnitude of change, and instructions foroutputting a confidence interval.
 2. The non-transitory machine-readablestorage medium for using prediction models for predicting values ofclaim 1, the medium further comprising: instructions for defining anentry time, the entry time being a first time when at least one of theplurality of entry identifiers was received.
 3. The non-transitorymachine-readable storage medium for using prediction models forpredicting values of claim 2, the medium further comprising:instructions for defining a time to change, the time to change being alag time between when at least one of the plurality of entry identifierswas received and when a change in the value for at least one of theplurality of entry identifiers was received.
 4. The non-transitorymachine-readable storage medium for using prediction models forpredicting values of claim 3, wherein the time to change is definea asD = {d₁, …  , d_(n_(t_(max)))} where d_(m)=min(argmax_(i)c_(m)i,t_(max))−e_(m).
 5. The non-transitory machine-readable storage mediumfor using prediction models for predicting values of claim 1, whereinthe change, C^(i) is determined by C^(i) = {c_(p_(j)^(i))} where$c_{p_{j}^{i}} = \left\{ {\begin{matrix}1 & {{{if}\mspace{14mu} v_{p_{j}^{i}}} \neq v_{p_{j}^{i + 1}}} \\0 & {else}\end{matrix},} \right.$ where v is the value for each of the pluralityof entry identifiers.
 6. The non-transitory machine-readable storagemedium for using prediction models for predicting values of claim 1,wherein the model for predicting a time-to-change model is a Poissonprocess model.
 7. The non-transitory machine-readable storage medium forusing prediction models for predicting values of claim 2, wherein themodel for predicting a time-to-change model is a Cox regression model.8. The non-transitory machine-readable storage medium for usingprediction models for predicting values of claim 1, wherein the modelfor predicting the time-to-change outputs a probability that the valuefor each of the plurality of entry identifiers will change.
 9. Thenon-transitory machine-readable storage medium for using predictionmodels for predicting values of claim 1, wherein the model forpredicting the future magnitude of change is a first order statisticmodel of the magnitude of change.
 10. The non-transitorymachine-readable storage medium for using prediction models forpredicting values of claim 1, wherein the magnitude of change, ΔV isdefined as ΔV={v_(p)(t_(m)+d_(m))−v_(p)t_(m)} for each of the pluralityof entry identifiers with c_(p)=1 in any C^(i), where v is the value foreach of the plurality of entry identifiers, c is the change and t is thetime value.
 11. The non-transitory machine-readable storage medium forusing prediction models for predicting values of claim 1, wherein thesimulation using the model for predicting the time-to-value change andthe model for predicting the future magnitude of change is a Monte Carlosimulation.
 12. The non-transitory machine-readable storage medium forusing prediction models for predicting values of claim 7, wherein thesimulation changes the magnitude of change using the probability thatthe value for each of the plurality of entry identifiers will change andthe magnitude of change.
 13. A method for providing predictions usingprediction models for predicting values, the method comprising the stepsof: receiving a plurality of entry identifiers; receiving a value foreach of the plurality of entry identifiers; determining whether thevalue for each of the plurality of entry identifiers has changed and amagnitude of the change; building a model for predicting a time-to-valuechange; building a model for predicting a future magnitude of change;performing a simulation using the model for predicting the time-to-valuechange and the model for predicting the future magnitude of change, andoutputting a confidence interval.
 14. The method for providingpredictions using prediction models for predicting values of claim 12,the method further comprising the step of: defining an entry time, theentry time being a first time when at least one of the plurality ofentry identifiers was received.
 15. The method for providing predictionsusing prediction models for predicting values of claim 13, the methodfurther comprising the step of: defining a time to change, the time tochange being a lag time between when at least one of the plurality ofentry identifiers was received and when a change in the value for atleast one of the plurality of entry identifiers was received.
 16. Themethod for providing predictions using prediction models for predictingvalues of claim 14, wherein the time to change is defined asD = {d₁, …  , d_(n_(t_(max)))} where d_(m)=min(argmax_(i)c_(m)i,t_(max))−e_(m).
 17. The method for providing predictions usingprediction models for predicting values of claim 12, wherein the change,C^(i) is determined by C^(i) = {c_(p_(j)^(i))} where$c_{p_{j}^{i}} = \left\{ {\begin{matrix}1 & {{{if}\mspace{14mu} v_{p_{j}^{i}}} \neq v_{p_{j}^{i + 1}}} \\0 & {else}\end{matrix},} \right.$ where v is the value for each of the pluralityof entry identifiers.
 18. The method for providing predictions usingprediction models for predicting values of claim 12, wherein the modelfor predicting a time-to-change model is a Poisson process model. 19.The method for providing predictions using prediction models forpredicting values of claim 12, wherein the model for predicting atime-to-change model is a Cox regression model.
 20. The method forproviding predictions using prediction models for predicting values ofclaim 12, wherein the model for predicting the time-to-change outputs aprobability that the value for each of the plurality of entryidentifiers will change.
 21. The method for providing predictions usingprediction models for predicting values of claim 12, wherein the modelfor predicting the future magnitude of change is a first order statisticmodel of the magnitude of change.
 22. The method for providingpredictions using prediction models for predicting values of claim 12,wherein the future magnitude of change, ΔV is defined asΔV={v_(p)(t_(m)+d_(m))−v_(p)t_(m)} for each of the plurality of entryidentifiers with c_(p)=1 in any C^(i), where v is the value for each ofthe plurality of entry identifiers, c is the change and t is the timevalue.
 23. The method for providing predictions using prediction modelsfor predicting values of claim 12, wherein the simulation using themodel for predicting the time-to-value change and the model forpredicting the future magnitude of change is a Monte Carlo simulation.24. The method for providing predictions using prediction models forpredicting values of claim 19, wherein the simulation changes themagnitude of change using the a probability that the value for each ofthe plurality of entry identifiers will change and the magnitude ofchange.